Communication exchange system for remotely communicating instructions

ABSTRACT

A server may communicatively couple to a user device and an instructor device. The server may receive location information from the user device. The location information may define visual content captured by the user device. The server may transmit the location information to the instructor device. The instructor device may present the visual content based on the received location information and receive input defining an instruction from an instructor. The server may receive instruction information defining the instruction from the instructor device. The server may transmit the instruction information to the user device. The user device may present the instruction overlaid on top of the visual content based on the received instruction information.

REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 16/713,492, filed on Dec. 13, 2019 and titled “COMMUNICATIONEXCHANGE SYSTEM FOR REMOTELY COMMUNICATING INSTRUCTIONS”, which claimspriority to U.S. patent application Ser. No. 15/993,976, filed May 31,2018, which has been granted as U.S. Pat. No. 10,511,881, issued on Dec.17, 2019 and titled “COMMUNICATION EXCHANGE SYSTEM FOR REMOTELYCOMMUNICATING INSTRUCTIONS,” which is incorporated herein by referencein its entirety.

TECHNICAL FIELD

The disclosed technology relates generally to communication exchangesystems, and in particular to technology for remotely communicatinginstructions.

BACKGROUND

In some instances, it may be necessary to communicate instructions to aremote person to enable the person to carry out some task. For instance,a medical emergency scenario at a location may require a non-medicalperson at the location to perform a medical procedure or to useunfamiliar medical equipment. A person trained on the medical procedureor the medical equipment may not be at the location and may need tocommunicate instructions relating to the medical procedure or themedical equipment to the person at the location.

BRIEF SUMMARY OF EMBODIMENTS

One aspect of the present disclosure is directed to a system forremotely communicating instructions. The system may comprise a serverthat communicatively couples to a first user device and an instructordevice. The server, the first user device, and the instructor device mayindividually comprise a memory and a processor. The server may beconfigured to: receive location information from the first user device,the location information characterizing a location of the first userdevice, wherein the location information defines visual content capturedby the first user device; transmit at least a portion of the locationinformation to the instructor device, the instructor device configuredto present the visual content within an instructor interface based onthe received location information and receive input from an instructorthrough the instructor interface, the input defining an instructionassociated with the visual content; receive instruction informationdefining the instruction associated with the visual content from theinstructor device; and transmit at least a portion of the instructioninformation to the first user device, the first user device configuredto present the instruction overlaid on top of the visual content withina first learner interface based on the received instruction information.

Another aspect of the present disclosure is directed to a method forremotely communicating instructions. The method may be performed by aserver communicatively coupled to a first user device and an instructordevice. The server, the first user device, and the instructor device mayindividually comprising a memory and a processor. The method maycomprise: receiving location information from the first user device, thelocation information characterizing a location of the first user device,wherein the location information defines visual content captured by thefirst user device; transmitting at least a portion of the locationinformation to the instructor device, the instructor device configuredto present the visual content within an instructor interface based onthe received location information and receive input from an instructorthrough the instructor interface, the input defining an instructionassociated with the visual content; receiving instruction informationdefining the instruction associated with the visual content from theinstructor device; and transmitting at least a portion of theinstruction information to the first user device, the first user deviceconfigured to present the instruction overlaid on top of the visualcontent within a first learner interface based on the receivedinstruction information.

Another aspect of the present disclosure is directed to a non-transitorycomputer readable medium comprising instructions that, when executed,may cause a server communicatively coupled to a first user device and aninstructor device to perform: receiving location information from thefirst user device, the location information characterizing a location ofthe first user device, wherein the location information defines visualcontent captured by the first user device; transmitting at least aportion of the location information to the instructor device, theinstructor device configured to present the visual content within aninstructor interface based on the received location information andreceive input from an instructor through the instructor interface, theinput defining an instruction associated with the visual content;receiving instruction information defining the instruction associatedwith the visual content from the instructor device; and transmitting atleast a portion of the instruction information to the first user device,the first user device configured to present the instruction overlaid ontop of the visual content within a first learner interface based on thereceived instruction information.

In some embodiments, the visual content may include one or more imagesor one or more videos of the location.

In some embodiments, the first learner interface may include a recordoption to record one or more portions of the instruction being presentedon top of the visual content. The first learner interface may include abookmark option to bookmark one or more portions of the instructionbeing presented on top of the visual content.

In some embodiments, the instructor interface may include a segmentationoption to segment the presentation of the instruction by the first userdevice into multiple parts. Presentation of different parts of theinstruction by the first user device may be controlled through the firstlearner interface or the instructor interface.

In some embodiments, the first learner interface may include a changeoption to change the instruction. The server may be further configuredto facilitate exchange of the change to the instruction between thefirst user device and the instructor device.

In some embodiments, the presentation of the instruction by the firstuser device may include a visual representation of a usage of an itemwith respect to a target at the location. The visual representation ofthe usage of the item with respect to the target at the location may bescaled based on a size of the target.

In some embodiments, the instructor interface may include an item optionto allow the instructor to interact with a visual representation of theitem to define the instruction on the usage of the item with respect tothe target at the location. The visual representation of the item ispresented in the instructor interface based on a determination that theitem is available for use at the location. The item option may include aset of preset options defining preset usage of the item with respect tothe target at the location. At least some of the set of preset optionsmay be included within the instructor interface based on the locationinformation further defining one or more characteristics of the targetat the location.

In some embodiments, the item option may allow the instructor to definethe usage of the item with respect to the target at the location basedon a three-dimensional model of the target. The visual representation ofthe usage of the item with respect to the target at the location may bepresented based on a three-dimensional model of the target. Athree-dimensional model of the target may be generated based on athree-dimensional mapping of the target by the first user device, andthe location information may further defines the three-dimensionalmapping of the target.

In some embodiments, the server may further communicatively couple to asecond user device. The server may be further configured to transmitcompanion instruction information to the second user device. Thecompanion instruction information may define the instruction. The seconduser device may be configured to present the visual representation ofthe usage of the item with respect to the target at the location withina second learner interface based on the received companion instructioninformation.

In some embodiments, the presentation the visual representation of theusage of the item with respect to the target at the location by thefirst user device and the second user device may include differentperspectives of the usage of the item with respect to the target at thelocation based on orientations of the first user device and the seconduser device with respect to the target at the location.

Other features and aspects of the disclosed technology will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, which illustrate, by way of example, thefeatures in accordance with embodiments of the disclosed technology. Thesummary is not intended to limit the scope of any inventions describedherein, which are defined solely by the claims attached hereto.

BRIEF DESCRIPTION OF THE DRAWINGS

The technology disclosed herein, in accordance with one or more variousembodiments, is described in detail with reference to the followingfigures. The drawings are provided for purposes of illustration only andmerely depict typical or example embodiments of the disclosedtechnology. These drawings are provided to facilitate the reader'sunderstanding of the disclosed technology and shall not be consideredlimiting of the breadth, scope, or applicability thereof. It should benoted that for clarity and ease of illustration these drawings are notnecessarily made to scale.

FIG. 1 illustrates an example environment in which one or moreembodiments in accordance with the technology of the present disclosuremay be implemented.

FIG. 2 illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein.

FIG. 3 illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein.

FIG. 4 illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein

FIG. 5A illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein.

FIG. 5B illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein.

FIG. 6 illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein.

FIG. 7 illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein.

FIG. 8 illustrates an example user interface in accordance with one ormore embodiments of the technology disclosed herein.

FIG. 9 illustrates an example method in accordance with one or moreembodiments of the technology disclosed herein.

FIG. 10 illustrates an example method in accordance with one or moreembodiments of the technology disclosed herein.

FIG. 11 illustrates an example method in accordance with one or moreembodiments of the technology disclosed herein.

FIG. 12 illustrates an example computing circuit that may be used inimplementing various features of embodiments of the technology disclosedherein.

The figures are not intended to be exhaustive or to limit the inventionto the precise form disclosed. It should be understood that theinvention can be practiced with modification and alteration, and thatthe disclosed technology be limited only by the claims and theequivalents thereof.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 shows an example environment 100 in which one or more embodimentsof the technology disclosed herein may be implemented. The environment100 may include multiple computing devices (e.g., a server 120, a userdevice 130A, a user device 130B, an instructor device 140), one or moreusers (e.g., a user 132A, a user 132B), one or more instructors (e.g.,an instructor 142), one or more targets (e.g., a target 160), one ormore items (e.g., items 170), and/or other components. The server 120may communicatively couple to one or more other computing devices, suchas the user device 130A, the user device 130B, and/or the instructordevice 140, through one or more networks 150. The server 120 may receivelocation information from at least one of the user devices 130A, 130B.For example, the server may receive location information from the userdevice 130A. The location information may characterize a location 110 ofthe user device 130A. The location 110 may also include one or more ofthe user device 130B, the user 132A, the user 132B, the target 160,and/or the items 170. The location information may define visual content(e.g., image(s), video(s)) captured by the user device 130A. The server120 may, responsive to receiving the location information from the userdevice 130A, transmit at least a portion of the location information tothe instructor device 140. The instructor device 140 may present thevisual content within an instructor interface (e.g., presented on adisplay of the instructor device 140) based on the received locationinformation. The instructor device 140 may receive input from theinstructor 142 through the instructor interface. The input from theinstructor 142 may define one or more instructions associated with thevisual content. The server may 120 receive instruction information fromthe instructor device 140. The instruction information may define theinstruction(s) associated with the visual content. The server 120 may,responsive to receiving the instruction information from the instructordevice 140, transmit at least a portion of the instruction informationto the user device 130A. The user device 130A may present theinstruction(s) overlaid on top of the visual content within a userinterface (e.g., presented on a display of the user device 130A) basedon the received instruction information.

Having thus described an example environment in which the disclosedtechnology can be implemented, various features and embodiments of thedisclosed technology are now described in further detail. After readingthe description herein, it will become apparent to one of ordinary skillin the art that the disclosed technology can be implemented in any of anumber of different environments.

A computing device within the environment 100 may refer to a machine forperforming one or more calculations, such as a computer, a dataprocessor, an information processing system, and/or other computingdevice. A computing device may include a mobile device, such as alaptop, a tablet, a smartphone, a smartwatch, smart glasses, a smartwear, a PDA, and/or other mobile device. A computing device may includea non-mobile device, such as a desktop computer and/or other non-mobiledevice. A computing device may include one or more processors, memory,and/or other components. The processor(s) may include one or morephysical processors and/or one or more virtual processors. Theprocessor(s) may be configured by software, firmware, and/or hardware toperform one or more functions described herein. Memory may includepermanent memory and/or non-permanent memory. Memory may store one ormore instructions, which may be executed by the processor(s). Theexecution of the instruction(s) by the processor(s) may cause thecomputing device to perform one or more functionalities describedherein. A computing device may include and/or be coupled with othercomponents, such as interface components, to perform its functions.

The server 120, the user devices 130A, 130B, the instructor device 140may communicate through the network(s) 150. The type of communicationfacilitated by through the network(s) 150 may vary, for example, basedon the communication protocol available to the server 120, the userdevices 130A, 130B, and/or the instructor device 140. Some non-limitingexamples of communication protocols over which the computing devices maycommunicate through the network(s) 150 may include: cellulartelecommunications protocols, such as GSM, UMTS, CDMA2000, LTE, orWiMAX; wired communication methods, such as cable, DSL, dial-up, orfiber-optic; and/or wireless communication methods, such as satellitecommunications, Wi-Fi, Bluetooth, or near-field communication (NFC).Usage of other communication protocols are contemplated.

The environment 100 may also include one or more datastores and/or oneor more databases that are accessible to one or more computing devices.The database(s) and/or datastore(s) may be stored within one or morememories of the computing devices, stored in one or more memories ofdevices coupled to a computing device, and/or may be accessible via oneor more networks, such as the network(s) 150.

While various computing devices (e.g., the server 120, the user device130A, the user device 130B, the instructor device 140) are shown in FIG.1 as single entities, this is merely for ease of reference and is notmeant to be limiting. One or more components/functionalities of acomputing device described herein may be implemented, in whole or inpart, within a single computing device or within multiple computingdevices. For example, the server 120 may refer to a single server,multiple servers that are physically co-located, and/or multiple serversthat are located in different physical locations.

The distribution of computing devices/functionalities shown in FIG. 1 ismerely an example and is not meant to be limiting. For example,components/functionalities of multiple computing devices may beimplemented within a single computing device. For instance, rather thanthe instructor device 140 being separate and apart from the server 120,the instructor device 140 may be part of the server 120 and/or locatedwith/near the server 120. Other distribution of computingdevices/functionalities are contemplated.

While the disclosure is described herein with respect to providinginstructions relating to medical procedures and operation of machines,this is merely for illustrative purposes and is not meant to belimiting. The approach disclosed herein may be used to provide othertypes of instructions to users.

The environment 100 may include the location 110. The location 110 mayrefer to a particular place or position. The location 110 may includethe users 132A, 132B, the user devices 130A, 130B, the target 160,and/or the items 170. The location 110 may not include the server 120,the instructor 142, and/or the instructor device 140.

The users 132A, 132B may refer to persons at the location 110. One ormore of the users 132A, 132B may require instructions from a person notat the location 110, such as the instructor 142. The user(s) 132A, 132Bmay require instructions relating to the target 160.

The user devices 130A, 130B may refer to computing devices associatedwith (e.g., used by, owned by, registered to) the users 132A, 132B. Theuser devices 130A, 130B may include mobile devices and/or non-mobiledevices. The user devices 130A, 130B may include one or more displays(e.g., touchscreen-display, non-touchscreen display), one or morespeakers, one or more microphones, and/or one or more interface devices(e.g., physical/virtual keyboard and/or mouse).

One or more of the user devices 130A, 130B may generate locationinformation for the location 110. Location information may refer toinformation that describes a location. That is, the location informationmay characterize the location 110. The location information maycharacterize physical and/or non-physical aspects of the location 110.For example, the location information may define geographical aspects ofthe location 110, such as the longitude, latitude, and/or elevation ofthe location 110, the address of the location 110, and/or thesurrounding of the location.

The location information may define one or more visual content (e.g.,images, videos) captured by one of the user devices 130A, 130B. Forexample, the user device 130A may include one or more cameras, includinglens(es) and image sensor(s), and the user 132A may use the user device130A to capture image(s) and/or video(s) of the location 110 as part ofthe location information. The image(s) and/or the video(s) of thelocation 110 may include visual representations of objects within thelocation 110, such as the target 160 and/or the items 170. The image(s)and/or the video(s) of the location 110 may be analyzed to determineadditional information about the location 110. For instance, visualanalysis of the image(s) and/or the video(s) may be performed toidentify the items 170 present at the location, to determinecharacteristics of the items 170, to identify the target 160, and/or todetermine characteristics of the target 170. The image(s) and/or thevideo(s) may be analyzed to determine a three-dimensional mapping and/ora three-dimensional model of an object at the location 110. For example,the user device 130A may include multiple cameras and may be used tocapture visual representations of the target 160 from multipleperspectives. The visual representations of the target 160 from multipleperspectives may be used, along with intrinsic and/or extrinsic cameraparameters, to determine a three-dimensional mapping of the target 160.The three dimensional mapping of the target 160 may include a depth mapthat contain information relating to the distance of the surfaces of thetarget from a viewpoint (e.g., the user device 130A). Thethree-dimensional mapping of the target 160 may be used to generate athree-dimensional model of the target 160. Other analysis of theimage(s) and/or the video(s) captured by the user device 130A arecontemplated. Analysis of the image(s) and/or video(s) may be performedby the user device 130A, the server 120, the instructor device 140,and/or other computing devices.

The location information may define one or more objects at the location110. Objects at the location 110 may refer to living and/or non-livingthings that are at the location 110. For example, the locationinformation may identify the target 160 and/or the items 170. Thelocation information may describe one or more characteristics of thetarget 160 and/or the items 170. The location information may define anobject at the location 110 based on user input and/or other information.For example, the user device 130A may generate location information thatidentifies and/or describes the characteristics of the target 160 basedon the user 132A entering information relating to the target 160 intothe device 130A. As another example, the user device 130A may generatelocation information that identifies and/or describes thecharacteristics of one or more of the items 170 based on visual analysisof the image(s)/video(s) captured by the user device 130A. Visualanalysis of the image(s)/video(s) may include visual analysis thatdetects features and/or shapes of the objects and matches the detectedfeatures/shapes with particular objects, visual analysis that identifiestags, such as QR codes, associated with particular objects, and/or othervisual analysis. Such location information may be used to determine thatparticular objects are at the location 110. For instance, such locationinformation may be used to determine that particular items (e.g., theitems 170) are available for use at the location 110. As furtherexample, the location information may define one or more characteristicsof the user 132A and/or the user 132B. For instance, the locationinformation may identify the user 132A and/or other information relatingto the user 132A that may be relevant to efforts to the directed to thetarget 160. For example, the education and/or vocation training of theuser 132A may be relevant for the instructor 142 in determining thedepth of instruction to be provided to the user 132A, and suchinformation may be generated by the user device 130A.

One or more of the user devices 130A, 130B may transmit the locationinformation to the server 120 (e.g., via the network(s) 150). Forexample, the user device 130A may capture image(s) and/or video(s) ofthe location 110 and may convey information defining the image(s) and/orthe video(s) to the server 120. The user device 130A may generatelocation information that describes the location and/or object(s) withinthe location and may provide the location information to the server 120.The location information may be transmitted by the user devices 130A,130B to the server 120 automatically and/or based on user input. Forinstance, the user 132A may start an application on the user device 130Ato receive instruction from the instructor 142. The application mayenable the user 132A to use the user device 130A to generate locationinformation for the location 110, such as by capturing image(s)/video(s)and/or by manually entering information into the user device 130A. Theapplication may send the location information to the server 120 as thelocation information is generated by the user device 130A and/or basedon the user 132A indicating to the application to send the locationinformation to the server 120.

The target 160 may refer to one or more living and/or non-living thingsfor which efforts are directed. The target 160 may refer to a thing forwhich one or both of the users 132A, 132B may require instructions. Forexample, the target 160 may refer to a person who requires a medicalprocedure and for whom the user 132A requires medical instructions fromthe instructor 142. As another example, the target may refer to amachine that needs to be operated and for which the user 132A requireoperating instructions from the instructor 142. Other types of targetsare contemplated.

The server 120 may refer to a computing device that facilitates exchangeof information between one or more of the user devices 130A, 130B andthe instructor device 140. The server 120 may include a mobile deviceand/or a non-mobile device. For instance, the server 120 may receiveinformation from the user device 130A and, responsive to the receptionof the information, relay the received information to the instructordevice 140, and vice versa. For example, the server 120 may receivelocation information from the user device 130A, and relay the locationinformation to the instructor device 140. The relay of informationperformed by the server 120 may include reception of information fromthe user device 130A and transmission of a copy of the receivedinformation to the instructor device 140. That is, the server 120 mayrelay an exact copy of the information received from the user device130A to the instructor device 140. For example, the server 120 mayreceive an image or a video of the location 110 from the user device130A and may transmit the image or the video to the instructor device140.

The relay of information performed by the server 120 may includereception of information from the user device 130A and transmission of amodified version of the received information to the instructor device140. For example, the server 120 may transmit one or more portions ofthe received information to the instructor device 140 or may alter oneor more portions of the received information before transmission to theinstructor device 140. For instance, the server 120 may receive an imageor a video of the location 110 from the user device 130A and maytransmit a lower fidelity version of the image or the video (e.g.,having lower resolution, having lower color depth, having lowerframerate) to the instructor device 140. Such modification of thelocation information may provide for resource savings, such as lowerbandwidth, lower processing cost, lower memory usage, and/or otherreduction of computing resources in facilitating exchange of informationbetween the user device 130A and the instructor device 140.

As further example, the server 120 may determine other information basedon the received information and transmit the other information to theinstructor device 140. For instance, based on the location informationidentifying the coordinate position and/or the address of the location110, the server 120 may access one or more databases that includeinformation on different items available for use at different locations.The server 120 may access such database(s) to determine a list of itemsavailable for use by the user 132A at the location 110, such as a listincluding the items 170. The server 120 may provide such list of itemsand/or information relating to items available for use at the location110 to the instructor device 140. Or, based on the location informationincluding images/videos of the items 170 or the target 160, the server120 may use visual analysis to identify the items 170 or the target 160and/or to determine characteristics of the items 170 or the target 160.Based on the location information including images with visualrepresentations of the target 160 from multiple perspectives, the server120 may determine a three-dimensional mapping of the target 160 and/orgenerate a three-dimensional model of the target 160. The server 120 mayprovide the three-dimensional mapping/model of the target 160 to theinstructor device 140. Provision of the three-dimensional mapping/modelof the target 160 to the instructor device 140 may enable the instructor142 to provide three-dimensional instruction for the target 160 and/orinstruction for the target 160 that takes into account the shape and/orsize of the target 160.

The instructor device 140 may refer to a computing device associatedwith (e.g., used by, owned by, registered to) the instructor 142. Theinstructor device 140 may include a mobile device and/or a non-mobiledevice. The instructor device 140 may include one or more displays(e.g., touchscreen-display, non-touchscreen display), one or morespeakers, one or more microphones, and/or one or more interface devices(e.g., physical/virtual keyboard and/or mouse). The instructor 140 mayrefer to a person remote from the location 110. The instructor 140 mayprovide instructions to a person at the location 110, such as the user132A or the user 132B. The instructor 142 may provide instructionsrelating to the target 160.

The instructor device 140 may receive at least a portion of the locationinformation from the server 120. The location information received bythe instructor device 140 from the server 120 may include a copy of thelocation information received by the server 120 (e.g., from the userdevice 130A), a modified version of the location information received bythe server 120, and/or other information determined by the server 120based on the location information received by the server 120. Theinstructor device 140 may, responsive to receiving information from theserver 120, provide a visual and/or verbal representation of theinformation to the instructor 142. For example, the instructor device140 may receive as/as part of location information visual content (e.g.,image, video) of the location 110 from the server 120, and theinstructor device 140 may present the visual content on a display of theinstructor device 140. The visual content may be presented within one ormore user interfaces based on the location information received from theserver 120. A user interface provided by the instructor device 140 maybe referred to as an instructor interface. A user interface provided bythe user devices 130A, 130B may be referred to as a learner interface.The instructor device 140 may receive input from the instructor 142through the instructor interface.

For example, based on the location information received from the server120, the instructor device 140 may present an image or a video of thelocation 110 within an instructor interface. The image/video of thelocation 110 may include a visual representation of the target 160.Based on the presentation of the target on a display of the instructordevice 140, the instructor 142 may provide input to the instructordevice 140. For example, the instructor 142 may use one or more of atouchscreen display, a microphone, a physical/virtual keyboard and/or amouse to provide input to the instructor device 140 through theinstructor interface.

The input from the instructor 142 may define one or more instructionsassociated with the visual content. For example, the input from theinstructor 142 may define a particular instruction to be provided to theuser 132A via the user device 130A in conjunction with the image/videowith which the instruction is associated. An instruction from theinstructor 142 may include a direction and/or an order for the user132A. An instruction from the instructor 142 may include informationthat details how the user 132A is to act with respect to the target 160.An act of the user 132A with respect to the target 160 may include theuser 132A acting alone, the user 132A acting with other person(s) (e.g.,the user 132B), and/or the user 132A using or not using one or moreitems (e.g., the items 170). The instruction from the instructor 142 maybe overlaid on top of an image or a video (associated with theinstruction) captured by the user device 130A. For example, theinstruction from the instructor 142 may be overlaid on top of animage/video captured by the user device 130A in the form of stationaryand/or moving image/graphic, text, message box, lines, and/or othervisual form.

For example, the target 160 may include a person needing medicalattention and an instruction from the instructor 142 may includeinformation on how the user 132A is to perform a medical procedure onthe target 160 and/or how the user 132A is to use one or more of theitems 170 on the target 160. As another example, the target 160 mayinclude a machine that requires to be operated and an instruction fromthe instructor 142 may include information on how the user is to operatethe target 160 and/or how the user 132A is to use one or more of theitems 170 on the target 160. The instruction from the instructor 142 maybe associated with image(s) or video(s) captured by the user device 130Asuch that the instruction is overlaid on presentation of the image(s) orthe video(s) by the user device 130A.

In some embodiments, the instructor interface may include one or moreitem options to allow the instructor 142 to interact with a visualrepresentation of an item to define an instruction on the usage of theitem with respect to the target 160 at the location 110. For example,the item option may include one or more options by which the instructor142 may select an item to be used and/or one or more options by whichthe instructor 142 may specify how the item is to be used with respectto the target 160. For instance, the instructor interface may presentvisual representations of various items (e.g., names, model number,icons, images, etc.), such as within an item drawer, in the instructorinterface for selection by the instructor 142.

In some embodiments, a visual representation of an item may be presentedin the instructor interface based on a determination that the item isavailable for use at the location 110. For example, an item may bedetermined to be available for use at the location 110 based on visualanalysis of an image/video captured by the user device 130A indicatingthe presence of the item at the location 110, such as among the items170, based on the user 132A indicating that the item is available,and/or based on the item being associated with the coordinate positionand/or the address of the location 110. Based on the determination thatthe item is available for use at the location 110, the instructor device140 may present the visual representation of item within the instructorinterface for selection by the instructor 142.

The instructor 142 may interact with the visual representation of theitem to define the instruction for the user 132A. For example, the itemmay include a tool and the visual representation of the item may includean image of the tool. The instructor 142 may move the image of the toolwithin the instructor interface and with respect to the visualrepresentation of the target 160 within the instructor interface todefine how the item is to be used on the target 160 by the user 132A. Insome embodiments, a visual representation of the usage of the item withrespect to the target 160 at the location 110 may be scaled based on asize of the target 160. That is, as the instructor moves the visualrepresentation of the target 160 within the instructor interface, thevisual representation of the target may increase or decrease in size toprovide an idea of scale of the item with respect to the target 160. Forinstance, the size of the visual representation of the item on theinstructor interface may change based on whether the target 160 is ababy or an adult. The instructor interface may include options for theinstructor 142 to manually change the scaling of the visualrepresentation of the item with respect to the target 160.

In some embodiments, the item option(s) provided by the instructorinterface may include one or more present options for selection by theinstructor 142. A preset option may define a preset usage of an itemwith respect to the target 160 at the location 110. For example, aparticular tool available at the location 110 may include a set ofpreset uses. The instructor interface may provide the preset uses ofthis tool so that the instructor 142 may be able to choose a presetusage for provision to the user 132A, rather than manually defining theusage of the tool. The item option(s) may also enable the instructor 142to change a preset usage of the tool. In such a case, the preset usageof the tool may be used as a starting point from which the instructor142 may determine the instruction to be provided to the user 132A.

In some embodiments, one or more of the preset options may be includedwithin the instructor interface based on the location informationfurther defining one or more characteristics of the target 160 at thelocation 110. For example, a particular tool available at the location110 may include a set of preset uses. Certain preset uses may only beappropriate for particular types of targets. For instance, theparticular tool may be a medical instrument and different preset optionsmay be defined for the tool based on the age of the target. In someembodiments, the preset options may be automatically modified based onthe characteristics of the target (e.g., as provided in the locationinformation). For example, the location information may provide theseverity of condition of the target 160 that must be addressed by theuser 132A, and the preset options presented on the instructor interfacemay be changed based on the severity of the condition of the target 160.

In some embodiments, the item option(s) provided by the instructorinterface may allow the instructor 142 to define a usage of an item withrespect to the target 160 at the location 110 based on athree-dimensional model of the target 160. A three-dimensional model ofthe target 160 may include a mathematical representation of the shapeand/or dimensions of the target 160. For instance, a three-dimensionalmodel of the target 160 may provide for the size, shape, curvature,and/or other physical characteristics of the surface(s) of the target160. A three-dimensional model of the target 160 may provide for how onepart of the target 160 may move with respect to the location 110 and/orwith other part(s) of the target 160. Defining a usage of an item withrespect to the target 160 based on a three-dimensional model of thetarget 160 may provide for instructions that takes into account theshape and/or dimensions of the target 160.

In some embodiments, the instructor interface may include a segmentationoption for presentation of the instruction to the user 132A by the userdevice 130A. The segmentation option may include one or more options tosegment the presentation of the instruction by the user device 130A intomultiple parts. For example, the instruction provided by the instructor142 may span a certain amount of time (e.g., two minutes), and thesegmentation option may include features that allow the instructor 142to segment the length of the instruction into multiple parts (e.g., abeginning part that spans the first thirty seconds, a middle part forthe following minute, a ending part that includes the last thirtyseconds of the instruction). Such segmentation of instructions mayenable the instructor 142 to separate a complex instruction intomultiple parts for the user 132A. Presentation of different parts of theinstruction by the user device 130A may be controlled through the userinterface (the learner interface) of the user device 130A and/or theinstructor interface. For example, the user 132A may interact with thelearner interface of the user device 130A to determine when the userwishes to proceed from seeing a beginning part of the instruction to thenext part of the instruction. As another example, presentation ofdifferent parts of the instructions may be controlled by the instructor142 through the instructor interface of the instructor device 140.Retaining control over the presentation of different parts of theinstruction may enable the instructor 142 to make sure that the user132A is following each part of the instruction and not gettingahead/skipping steps.

The instructor device 140 may transmit instruction information to theserver 120 (e.g., via the network(s) 150). The instruction informationmay define the instruction associated with the visual content. Theinstruction information may be transmitted to the server 120 for theserver 120 to relay the instruction information to the user device 130A.The instruction information may be transmitted by the instructor device140 to the server 120 automatically and/or based on instructor input.For instance, as the instructor 142 is providing input defininginstruction to the instructor device 140, the instructor device 140 maytransmit the instruction information to the server 120. As anotherexample, the instructor device 140 may transmit the instructioninformation to the server 120 once the instructor 142 indicates (via theinstructor interface) that the instruction should be sent.

The server 120 may receive the instruction information from theinstructor device 140. Responsive to reception of the instructioninformation, the server may relay the received instruction informationto one or both of the user devices 130A, 130B. The relay of theinstruction information performed by the server 120 may includetransmission of an exact copy of the instruction information receivedfrom the instructor device 140 to the user device(s) 130A, 130B,transmission of a modified version of the instruction informationreceived from the instructor device 140 to the user device(s) 130A,130B, and/or transmission of other information (e.g., informationdetermined based on the received instruction information) to the userdevice(s) 130A, 130B.

One or both of the user devices 130A, 130B may receive at least aportion of the instruction information from the server 120. The receivedinstruction information may be used to visually and/or verbally providethe instruction from the instructor 142 to the user(s) 132A, 132B. Forexample, the user device 130A may, based on the received instructioninformation, overlay the instruction on top of visual content (e.g.,image(s), video(s)) captured by the user device 130A. The instructionmay be presented on top of the visual content within the learnerinterface of the user device 130A. Overlaying of the instruction on topof the visual content may include placement of the instruction on alayer that is on top of the layer of the visual content and/or mayinclude insertion of the instruction into the visual content. Forinstance, the instruction and the visual content may be separate visualelements which are presented together. The instruction and the visualcontent may form a single visual element (e.g., a single encoded streamof image(s)/video(s)) what is presented. Overlaying of the instructionon top of the visual content may provide an augmented reality view ofthe scene captured within the visual content. For example, a view of thetarget 160 may be augmented with instructions from the instructor 142.In some embodiments, the presentation of the instruction by the userdevice 130A may include a visual representation of a usage of an item(e.g., one of the items 170) with respect to the target 160 at thelocation 110. The visual representation of the usage of the item withrespect to the target 160 may be scaled based on a size of the target.The visual representation of the usage of the item with respect to thetarget 160 may be presented based on a three-dimensional model of thetarget 160.

In some embodiments, the learner interface of the user device 130A mayinclude one or more record options to record one or more portions of theinstruction being presented on top of the visual content. For example,as the instruction from the instructor 142 is being presented on top ofthe visual content captured by the user device 130A, a record option maybe presented on the learner interface of the user device 130A. The user132A may interact with the record option to indicate to the user device130A that the user 132A wishes to record one or more portions of theinstruction presentation. Such recordation of the instruction may enablethe user 132A to preserve one or more portions or the entirety of theinstruction for replay. That is, after the instruction from theinstruction 142 has been presented on the display of the user device130A, the user 132A may replay the portion(s) the user 132A “recorded”on the user device 130A. In some embodiments, the user device 130A mayautomatically record the entire presentation of the instruction or oneor more particular portions of the instruction presentation. Forexample, when the instruction is received and/or presented on the userdevice 130A, the user device may automatically start to record thepresentation.

In some embodiments, the learner interface of the user device 130A mayinclude one or more bookmark options to bookmark one or more portions ofthe instruction being presented on top of the visual content. Forexample, as the instruction from the instructor 142 is being presentedon top of the visual content captured by the user device 130A, abookmark option may be presented on the learner interface of the userdevice 130A. The user 132A may interact with the bookmark option toindicate to the user device 130A that the user 132A wishes to mark oneor more portions of the instruction presentation. Such marking of theinstruction may enable the user 132A to preserve the location ofparticular moments within the instruction presentation. That is, duringreplay of the instruction on the user device 130A, the user 132A mayjump to a particular moment within the instruction presentation by usingthe bookmark marked at the moment. In some embodiments, one or morebookmarks may be created by the instructor 142 via the instructorinterface.

In some embodiments, the learner interface may include one or morechange options to change the instruction from the instructor 142. Forexample, as the instruction from the instructor 142 is being presentedon top of the visual content captured by the user device 130A, a changeoption may be presented on the learner interface of the user device130A. The user 132A may interact with the change option to indicate tothe user device 130A that the user 132A wishes to change one or moreportions of the instruction from the instructor 142. For example, theuser 132A may wish to change the instruction based on the instructionnot being clear and/or the user 132A not being able to perform theinstruction. The server 120 may facilitate exchange of the change(s) tothe instruction between the user device 130A and the instructor device140 by relaying information relating to the changes between the userdevice 130A and the instructor device 140.

In some embodiments, the instruction from the instructor 142 may beoverlaid on presentation of visual content within multiple user devices,such as the user device 130A and the user device 130B. The presentationof the instruction overlaid on top of the visual content may be the sameor different for the user devices 130A, 130B. In some embodiments, theserver 120 may transmit the same instruction information to both userdevices 130A, 130B and the user devices 130A, 130B may display the sameview of the instruction from the instructor 142. In some embodiments,the server 120 may transmit the same instruction information to bothuser devices 130A, 130B and the user devices 130A, 130B may displaydifferent views of the instruction from the instructor 142. In someembodiments, the server 120 may transmit the different instructioninformation to user devices 130A, 130B and the user devices 130A, 130Bmay display different views of the instruction from the instructor 142.

The user device 130B may provide a companion view of the instructionfrom the instructor 142. The instruction information transmitted by theserver 120 to the user device 130B may be referred to as companioninstruction. The companion instruction information may define theinstruction from the instructor 142. The user device 130B may use thereceived companion instruction information to visually and/or verballyprovide the instruction from the instructor 142. For example, the userdevice 130B may, based on the received companion instructioninformation, overlay the instruction on top of visual content (e.g.,image(s), video(s)) within a user interface of the user device 130B. Thevisual content may be captured by the user device 130A and/or the userdevice 130B. That is, the user device 130B may present the instructionoverlaid on top of image(s)/video(s) captured by the user device 130Aand/or may present the instruction overlaid on top of image(s)/video(s)captured by the user device 130B. In some embodiments, the companioninstruction information may define the visual content captured by theuser device 130A. Such presentation of a companion view may enable theuser 132B to watch over the execution of the instruction by the user132A and/or to ensure that the instruction is followed by the user 132A.

In some embodiments, the companion view presented by the user device130B may show a different perspective of the target 160 and/or theinstruction than the view of the instruction/the target 160 presented bythe user device 130A. For example, a companion view presented by theuser device 130B may include presentation of different perspective of ausage of an item with respect to the target 160 than presented on theuser device 130A. For example, the usage of the item with respect to thetarget 160 may be presented on the user device 130A based on visualcontent of the target 160 captured from the left side of the target 160.The usage of the item with respect to the target 160 may be presented onthe user device 130B based on visual content of the target 160 capturedfrom the right side of the target 160. That is, the differentperspectives of the usage of the item with respect to the target 160and/or other views of the instruction may be presented based on theorientations of the user devices 130A, 130B with respect to the target160 at the location 110.

FIGS. 2, 3, 4, 5A, 5B, 6, 7, and 8 illustrate example user interfaces200, 300, 400, 500, 550, 600, 700, 800 in accordance with one or moreembodiments of the technology disclosed herein. In various embodiments,the user interfaces 200, 300, 400, 500, 550, 600, 700, 800 may beaccessed through a software application running on a computing device(e.g., computers, mobile phones, tablets, etc.) that includes one ormore processors and memory. Depending on the computing device, a usermay be able to interact with the user interfaces 200, 300, 400, 500,550, 600, 700, 800 using various input devices (e.g., keyboard, mouse,etc.) and/or touch gestures. The user interfaces 200, 300, 400, 500,550, 600, 700, 800 are provided merely as examples and, naturally, thearrangement and configuration of such user interfaces can vary dependingon the implementation. Thus, depending on the implementation, the userinterfaces 200, 300, 400, 500, 550, 600, 700, 800 may include additionalfeatures and/or alternative features. The user interfaces 200, 300, 400,500, 550, 600, 700, 800 may include/enable one or more functionalitiesof the interface(s) described with respect to the user devices 130A,130B, the instructor device 140, and/or other components of theenvironment 100 described with respect with FIG. 1.

FIG. 2 illustrates an example instructor interface 200 in accordancewith one or more embodiments of the technology disclosed herein. Theinstructor interface 200 may be presented on and/or by an instructordevice, such as the instructor device 140. Visual content (e.g.,image(s), video(s)) may be presented within the instructor interface200. For example, as shown in FIG. 2, the instructor interface 200 mayinclude presentation of an image/video of a target 230. The target 230may be located remotely from the instructor device presenting theinstructor interface 200. The visual content presented within theinstructor interface 200 may be captured by a user device at thelocation of the target 230. Information defining the visual content maybe relayed to the instructor device over a network/server.

An instructor may interact with the instructor interface 200 to provideinstruction(s) to a person at the location of the target 230. Forexample, the instructor may interact with the instructor interface 200to provide instruction(s) to the user of the user device that captureimage(s)/video(s) of the target 230. For instance, the instructor maymove one or more icons on the instructor interface 200, such as iconsrepresenting items available for use at the location of the target 230,to show how the items are to be used with respect to the target 230. Theinstructor may interact with the instructor interface 200 to provideinstruction(s) that do not require the use of item(s). For example, theinstructor may use one or more icons on the instructor interface 200 toprovide instruction on how to interact with the target 230, such asdirections 242, 244. The direction 242 may include an arrow thatindicates that the user is to push up on the left side of the target230. The direction 244 may include an arrow that indicates that the useris to move a component from the top right side of the target 230 to thefront of the target 230. Other provision of instructions arecontemplated.

As another example, the instructor may use one or more icons on theinstructor interface 200 to provide instruction on how to perform CPR toa person. The instruction provided by using the instructor interface 200may include static objects or dynamic objects. For example, aninstruction on how to perform CPR may include static images representinghands of the person performing CPR that are placed on a particularlocation of the person receiving CPR (target). As another example, aninstruction on how to perform CPR may include dynamic imagesrepresenting hands of the person performing CPR that are placed on aparticular location of the target. The images representing the hands maychange (e.g., change in size, change in color, change in brightness) toindicate when the person performing CPR is to press down, to indicatethe pressure to be applied, and/or to otherwise convey certain aspectsof CPR instruction. In some embodiments, one or more portions of theinstructions may be provided visually, verbally (e.g., user deviceoutputting sounds/commands), and/or haptically.

The instructor interface 200 may include one or more options 210 for useby the instructor. For example, the options 210 may include zoom options212, shape options 214, line options 216, a free draw option 218, a textinsertion option 220, a drawer option 222, a segment option 224, abookmark option 226, a speed option 228, and/or other options. The zoomoptions 212 may enable the instructor to change the viewing zoom withwhich the visual content is displayed within the instructor interface212. The instructor may use the zoom options 212 to focus in/out on therelevant parts of the target 230 and/or the environment of the target230. For example, based on the visual content including a visualrepresentation of the entire target 230 and the instruction to beprovided being applicable to a small portion of the target 230, theinstructor may use the zoom options 212 to zoom in on the relevantportion and provide detailed instructions for the portion.

The shape options 214 may enable the instructor to insert one or moreshapes as part of the instruction. The line options 216 may enable theinstructor to insert one or more lines as part of the instruction. Thefree draw option 218 may enable the instructor to freely draw within theinstructor interface 200 to provide the instruction. The text insertionoption 220 may enable the instructor to insert text to provide theinstruction. The drawer option 222 may enable the instructor to see alist of items that are available to be used at the location of thetarget 230. The instructor may select one or more icons representingitems from the list and move the icons within the instructor interface200 to specify how the item(s) are to be used with respect to thetarget. The instructor may use one or more of the options 214, 216, 218,220, 222 to define the instruction to be overlaid on top of the visualcontent.

The segment option 224 may enable the instructor to segment thepresentation of the instruction by a user device into multiple parts.For example, the instructor may have define the direction 242 as thefirst part of the instruction. Before defining the direction 244 as thesecond part of the instruction, the instructor may use the segmentoption 224 to separate the second part of the instruction from the firstpart of the instruction. As another example, the instructor may, byinteracting with the instructor interface 200, define an instructor thatspans a certain amount of time. The instructor may use the segmentoption 224 while going over the instruction (e.g., replaying theinstruction, moving over different portions of the instruction using aseekbar) to segment the instructions by time. The bookmark option 226may enable the instructor to mark one or more particular moments withinthe instructions. Such marks may be used by the instructor and/or theuser to jump to the particular moments within the instructions.

The speed option 228 may enable the instructor to change the playbackspeed of the instruction. For example, the instruction defined by theinstructor using the instructor interface 200 may be presented on a userdevice at the same speed with which the instructor defined theinstruction. For instance, the instructor may have defined the direction242 by drawing the arrow over two seconds. The direction 242 may bepresented on the user device as being drawn over two seconds. Theinstructor may use the speed option 228 to change the playback speed ofone or more portions of the instruction. For example, the playback speedof the direction 242 may be increased so that it is shown on the userdevice more rapidly than it was defined within the instruction interface200, while the playback speed of the direction 244 may be decreased sothat it is shown on the user device more slowly than it was definedwithin the instruction interface 200.

FIG. 3 illustrates an example instructor interface 300 in accordancewith one or more embodiments of the technology disclosed herein. Theinstructor interface 300 may be presented on and/or by an instructordevice, such as the instructor device 140. Visual content may bepresented within the instructor interface 300. For example, as shown inFIG. 3, the instructor interface 300 may include presentation of animage/video of a target 330. The target 330 may be located remotely fromthe instructor device presenting the instructor interface 300. Thevisual content presented within the instructor interface 300 may becaptured by a user device at the location of the target 330. Informationdefining the visual content may be relayed to the instructor device overa network/server.

The instructor interface 300 may include one or more options 310 for useby the instructor. For example, the options 310 may include zoom options312, shape options 314, line options 316, a free draw option 318, a textinsertion option 320, a drawer option 322, a segment option 324, abookmark option 326, a speed option 328, and/or other options. Theoptions 310 may work as the options 210 described with respect to theinstructor interface 200. An instructor may interact with the instructorinterface 300 to provide instruction(s) to a person at the location ofthe target 330. For example, the instructor may interact with theinstructor interface 300 to provide instruction(s) to the user of theuser device that capture image(s)/video(s) of the target 330. Forexample, the instructor may use one or more icons on the instructorinterface 300 to provide instruction on how to interact with the target330, such as directions 342, 344. The direction 342 may include an arrowthat indicates that the user is to push up on the left side of thetarget 330. The direction 344 may include an arrow that indicates thatthe user is to move a component from the top right surface of the target330 to the front surface of the target 230. Other provision ofinstructions are contemplated.

The instructor interface 300 may allow the instructor to defineinstruction, such as a usage of an item, with respect to the target 330based on a three-dimensional model of the target 330. Thethree-dimensional model of the target 330 may provide for the size,shape, curvature, and/or other physical characteristics of thesurface(s) of the target 330. The three-dimensional model of the target330 may provide for how one part of the target 330 may move with respectto the location of the target 330 and/or with other part(s) of thetarget 330. Defining instructions with respect to the target 330 basedon a three-dimensional model of the target 330 may provide forinstructions that takes into account the shape and/or dimensions of thetarget 330. For instance, the direction 344 may be defined with respectto the three-dimensional model of the target 330 such that the beginningof the direction 344 is pinned to the center of the top-right surface ofthe target 330 and the ending of the direction 344 is pinned to the topright corner of the front surface of the target 330. Thethree-dimensional relationship between the target 330 and the direction344 may be preserved when the view of the target 330 changes. Forexample, the user device capturing the visual content of the target 330may change to capture a different perspective of the target 330, and thedirection 344 may move/change shape with the change in perspective topreserve the three-dimensional relationship between the target 330 andthe direction 344.

FIG. 4 illustrates an example instructor interface 400 in accordancewith one or more embodiments of the technology disclosed herein. Theinstructor interface 400 may be presented on and/or by an instructordevice, such as the instructor device 140. Visual content may bepresented within the instructor interface 400. For example, as shown inFIG. 4, the instructor interface 400 may include presentation of animage/video of a target 430. The target 430 may be located remotely fromthe instructor device presenting the instructor interface 400. Thevisual content presented within the instructor interface 400 may becaptured by a user device at the location of the target 430. Informationdefining the visual content may be relayed to the instructor device overa network/server.

The instructor interface 400 may include one or more options 410 for useby the instructor. For example, the options 410 may include zoom options412, shape options 414, line options 416, a free draw option 418, a textinsertion option 420, a drawer option 422, a segment option 424, abookmark option 426, a speed option 428, and/or other options. Theoptions 410 may work as the options 210 described with respect to theinstructor interface 200. An instructor may interact with the instructorinterface 400 to provide instruction(s) to a person at the location ofthe target 430. For example, the instructor may interact with theinstructor interface 400 to provide instruction(s) to the user of theuser device that capture image(s)/video(s) of the target 430. Forexample, the instructor may use one or more icons on the instructorinterface 400 to provide instruction on how to interact with the target430, such as directions 442, 444.

The instructor's use of the instructor interface 400 may change one ormore visual aspects of the visual content. For example, based on theinstructions defined by the instructors (e.g., the directions 442, 444)being focused on the front surface of the target 430, the front surfaceof the target 430 may be emphasized over others portions of the target430 within the instructor interface 400. The front surface of the target430 may be emphasized over others portions of the target 430 within thelearner interface presenting the directions 442, 444. For instance, thefront surface of the target 430 may be presented in color while otherportions of the target may be presented in gray scale. The front surfaceof the target 430 may be shown in focus while other portions of thetarget may be blurred. Other emphasis of the relevant portions of thetarget 430 are contemplated. In some embodiments, the emphasis of thedifferent portions of the target 430 may be performed using athree-dimensional model of the target 430. For example, thethree-dimensional model of the target 430 may be used to identify theportion(s) (e.g., surface(s)) of the target 430 at which instruction isdirected and to emphasize the corresponding portion of the image/video.In some embodiments, the options 410 may include an emphasis option thatenables the instructor to define which portions of the target 430 areemphasized/deemphasized.

FIG. 5A illustrates an example user interface 500 in accordance with oneor more embodiments of the technology disclosed herein. The userinterface 500 may be presented on and/or by an instructor device, suchas the instructor device 140, and/or a user device, such as the userdevice 130A or the user device 130B. The user interface 500 may bepresented within another interface, such as the instructor interfaces200, 300, 400 and/or a learner interface. For example, the userinterface 500 may be presented in response to a user or an instructorinteracting with an option, such as a drawer option. The user interfacemay provide a list of items that are available to be used at aparticular location. An instructor may select one or more iconsrepresenting the items from the list and move the icon(s) within theinstructor interface to specify how the item(s) are to be used withrespect to a target. A user may select one or more icons representingthe items from the list to see instructions (e.g., basic instructions)for using the item(s).

For example, the user interface 500 may include a portion 502 and aportion 504. The portion 502 may include icons 512, 514, 516, 518representing different items that are available to be used at alocation. An instructor and/or a user may select an icon, such as theicon 514, to see additional information relating to the correspondingitem. For example, responsive to selection of the icon 514 in theportion 502, additional information about the corresponding item may bedisplayed in the portion 504. For instance, responsive to selection ofthe icon 514, options 522, 524, 526, and 532 may be presented in theportion 504. The options 522, 524, 526 may enable a user or aninstructor to see different information relating to the itemcorresponding to the icon 514, such as one or more preset usages of theitem and/or instructions on how to use the item. The option 532 mayinclude a slider 534, which may be moved to change the scaling of theitem with respect to a target. For example, the slider 534 may be movedto one side to decrease the size of the item with respect to a targetand may be moved to the other side to increase the size of the item withrespect to the target.

FIG. 5B illustrates an example user interface 550 in accordance with oneor more embodiments of the technology disclosed herein. The userinterface 550 may be presented on and/or by an instructor device, suchas the instructor device 140, and/or a user device, such as the userdevice 130A or the user device 130B. The interface 550 may present aninstruction for using an item 562 with respect to a target 530. The item562 may have been selected from a list of item available to be used atthe location of the target 530, such as from the interface 500. The item562 may be moved within the instruction interface to define one or moreinstructions on the use of the item 562 with respect to the target 530.For example, an instructor may define the directions 564A, 564B for theitem 562 using the instructor interface, and the directions 564A, 564Bmay be overlaid on top of an image/video of the target 530 within alearner interface. The visual characteristics of the directions 564A,564 may be used to convey different instructions. For instance, thecolor, shape, and/or the size of the direction 564A may be changed toindicate the pressure and/or the speed with which the item 562 is to bemoved across the front surface of the target 530. As another example,the color, shape, and/or the size of the direction 564B may be changedto indicate the pressure, the length of time, and/or the depth withwhich the item 452 is to be pushed on/into the front surface of thetarget 530. For instance, the instruction may include informationindicating that the item 562 is to be contacted with the front surfaceof the target 530. The direction 564B may pulse and/or change color toindicate to a user when the item 562 is to be contacted with the frontsurface of the target 530. Other changes in visual characteristics ofinstructions and other types of instructions are contemplated.

FIG. 6 illustrates an example learner interface 600 in accordance withone or more embodiments of the technology disclosed herein. The learnerinterface 600 may be presented on and/or by a user device, such as theuser device 130A. Visual content may be presented within the learnerinterface 600. For example, as shown in FIG. 6, the learner interface600 may include presentation of an image/video of a target 630. Thetarget 630 may be located at the location of the user and the userdevice. The visual content presented within the learner interface 600may be captured by the user device at the location of the target 630.Information defining instructions associated with the visual content maybe relayed (e.g., over a network/server) to the user device from aninstructor device. For example, the instruction associated with thevisual content may include directions 642, 644. The directions may beoverlaid on top of the visual content and may provide instructions onhow to interact with the target 630.

The learner interface 600 may include one or more options 610 for use bythe user. For example, the options 610 may include a record option 612,a bookmark option 614, a speed option 616, a change option 618, and/orother options. The record option 612 may enable a user to record one ormore portions of the instruction presented within the learner interface600. For example, the user may toggle the record option 612 to turnon/off the recording of instructions presented on the learner interface600. In some embodiments, the “recording” of instructions may includemarking of one or more portions of the instructions. For example, theuser may interact with the record option 612 to mark certain portions ofthe instruction presentation for replay and/or review. The bookmarkoption 614 may enable the user to mark one or more particular momentswithin the instructions. Such marks may be used by the user and/or theinstructor to jump to the particular moments within the instructions.The speed option 616 may enable the user to change the playback speed ofthe instruction. For example, the user may user the speed option 616 toincrease and/or decrease the speed with which the instruction ispresented within the learner interface 600.

The change option 618 may enable the user to change one or more portionsof the instruction. For example, the user may insert text into theinstruction being presented within the learner interface 600 to ask theinstructor one or more questions. As another example, the user maychange the shape, size, direction, and/or other aspects of thedirections 642, 644. The information describing the changes made by theuser may be transmitted to the instructor device.

FIG. 7 illustrates an example learner interface 700 in accordance withone or more embodiments of the technology disclosed herein. The learnerinterface 700 may be presented on and/or by a user device, such as theuser device 130A. Visual content may be presented within the learnerinterface 700. For example, as shown in FIG. 7, the learner interface700 may include presentation of an image/video of a target 730. Thetarget 730 may be located at the location of the user and the userdevice. The visual content presented within the learner interface 700may be captured by the user device at the location of the target 730.Information defining instructions associated with the visual content maybe relayed (e.g., over a network/server) to the user device from aninstructor device. For example, the instruction associated with thevisual content may include directions 742, 744. The directions may beoverlaid on top of the visual content and may provide instructions onhow to interact with the target 730.

The learner interface 700 may include one or more options 710 for use bythe user. For example, the options 710 may include a record option 712,a bookmark option 714, a speed option 716, a change option 718, and/orother options. The options 710 may work as the options 610 describedwith respect to the instructor interface 600. The instruction providedby the instructor may be segmented into multiple parts. For example, aprogress bar 750 may be displayed on the learner interface 700. Theprogress bar may indicate the length of the instruction and whatmoment/duration of the instruction is being presented within the learnerinterface 700. For instance, as shown in FIG. 7, a black portion 752 mayindicate the portion of the instruction that has been presented. Thepresentation of the black portion 752 of the instruction may include thedirection 742 and may not include the direction 744. The direction 744may not be displayed and/or may be displayed different until thecorresponding portion of the instruction is reached. For example, thedirection 744 may be displayed in outline until the correspondingportion of the instruction is reached, at which point the direction 744may be displayed in full. The instruction may be segmented into twoparts by a divider 754. In some embodiments, presentation of differentparts of the instruction within the learner interface 700 may becontrolled by the instructor (e.g., through the instructor interface).For example, the user may only be shown the second part of theinstruction when the instructor gives access to the second part of theinstruction and/or when the instructor prompts the second part of theinstruction to be presented within the learner interface 700. Theinstruction may also be bookmarked, such as shown by a bookmark 756. Theuser and/or the instructor may use the bookmark 756 to jump to a momentin the second part of the instruction.

FIG. 8 illustrates an example learner interface 800 in accordance withone or more embodiments of the technology disclosed herein. The learnerinterface 800 may be presented on and/or by a user device, such as theuser device 130B. Visual content may be presented within the learnerinterface BOO. For example, as shown in FIG. 8, the learner interface800 may include presentation of an image/video of a target 830. Thetarget 830 may be located at the location of the user and the userdevice. The visual content presented within the learner interface 800may be captured by the user device at the location of the target 830.Information defining instructions associated with the visual content maybe relayed (e.g., over a network/server) to the user device from aninstructor device. For example, the instruction associated with thevisual content may include directions 842, 844. The directions may beoverlaid on top of the visual content and may provide instructions onhow to interact with the target 830. The learner interface 800 mayinclude one or more options 810 for use by the user. For example, thelearner interface 800 may include one or more options described withrespect to the options 610, and/or other options.

The learner interface 800 may provide a different view of theinstruction from an instructor based on changes in orientation of theuser device with respect to the target. For example, the user device maybe, based on an original position of the user device with respect to thetarget, be presenting a view of the target as shown in FIG. 6. Based onchanges in the position of the user device and/or the target, the viewof the target as shown in FIG. 8 may be displayed. The instructions maybe changed to account for the change in perspective of the target shownwithin the learner interface 800.

The learner interface 800 may provide a companion view of theinstruction from an instructor. For example, there may be two userdevices at the location of the target 830. One user device may bepositioned to see a view of the target as shown in FIG. 6. The otheruser device may be positioned to see a view of the target as shown inFIG. 8. Thus, the two user devices may see differentversions/perspectives of the instruction based on the orientations ofthe user devices with respect to the target.

FIG. 9 illustrates an example method 900 that may be implemented inaccordance with one or more embodiments of the technology disclosedherein. The steps of the method 900 may be implemented in/through one ormore computing devices, such as the server 120 (as shown in FIG. 1 anddescribed herein). At step 902, location information may be receivedfrom a user device. At step 904, at least a portion of the locationinformation may be transmitted to an instructor device. At step 906,instruction information defining an instruction may be received from theinstructor device. At step 908, at least a portion of the instructioninformation may be transmitted to the user device.

FIG. 10 illustrates an example method that may be implemented inaccordance with one or more embodiments of the technology disclosedherein. The steps of the method 1000 may be implemented in/through oneor more computing devices, such as the user device 130A and/or the userdevice 130B (as shown in FIG. 1 and described herein). At step 1002,location information for a location may be generated. At step 1004, atleast a portion of the location information may be transmitted to aserver. At step 1006, instruction information defining an instructionmay be received from the server. At step 1008, the instruction may bepresented overlaid on top of visual content.

FIG. 11 illustrates an example method that may be implemented inaccordance with one or more embodiments of the technology disclosedherein. The steps of the method 1100 may be implemented in/through oneor more computing devices, such as the instructor device 140 (as shownin FIG. 1 and described herein). At step 1102, location information maybe received from a server. At step 1104, visual content defined by thelocation information may be presented. At step 1106, input defining aninstruction associated with the visual content may be received. At step1108, the instruction information defining the instruction may betransmitted to the server.

As used herein, the term circuit/logic might describe a given unit offunctionality that can be performed in accordance with one or moreembodiments of the technology disclosed herein. As used herein, acircuit/logic might be implemented utilizing any form of hardware,software, firmware, or a combination thereof. For example, one or moreprocessors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logicalcomponents, software routines or other mechanisms might be implementedto make up a circuit/logic. In implementation, the variouscircuits/logics described herein might be implemented as discretecircuits/logics or the functions and features described can be shared inpart or in total among one or more circuits/logics. In other words, aswould be apparent to one of ordinary skill in the art after reading thisdescription, the various features and functionality described herein maybe implemented in any given application and can be implemented in one ormore separate or shared circuits/logics in various combinations andpermutations. Even though various features or elements of functionalitymay be individually described or claimed as separate circuits/logics,one of ordinary skill in the art will understand that these features andfunctionality can be shared among one or more common software andhardware elements, and such description shall not require or imply thatseparate hardware or software components are used to implement suchfeatures or functionality.

Where components or circuits/logics of the technology are implemented inwhole or in part using software, in one embodiment, these softwareelements can be implemented to operate with a computing or processingcircuit/logic capable of carrying out the functionality described withrespect thereto. One such example computing circuit/logic is shown inFIG. 12. Various embodiments are described in terms of thisexample-computing circuit 1200. After reading this description, it willbecome apparent to a person skilled in the relevant art how to implementthe technology using other computing circuits/logics or architectures.

Referring now to FIG. 12, computing circuit 1200 may represent, forexample, computing or processing capabilities found within desktop,laptop and notebook computers; hand-held/wearable computing devices(PDA's, smart phones, smart glasses, cell phones, palmtops, etc.);mainframes, supercomputers, workstations or servers; or any other typeof special-purpose or general-purpose computing devices as may bedesirable or appropriate for a given application or environment.Computing circuit 1200 might also represent computing capabilitiesembedded within or otherwise available to a given device. For example, acomputing circuit might be found in other electronic devices such as,for example, digital cameras, navigation systems, cellular telephones,portable computing devices, modems, routers, WAPs, terminals and otherelectronic devices that might include some form of processingcapability.

Computing circuit 1200 might include, for example, one or moreprocessors, controllers, control circuits, or other processing devices,such as a processor 1204. Processor 1204 might be implemented using ageneral-purpose or special-purpose processing engine such as, forexample, a microprocessor, controller, or other control logic. In theillustrated example, processor 1204 is connected to a bus 1202, althoughany communication medium can be used to facilitate interaction withother components of computing circuit 1200 or to communicate externally.

Computing circuit 1200 might also include one or more memory components,simply referred to herein as main memory 1208. For example, preferablyrandom access memory (RAM) or other dynamic memory, might be used forstoring information and instructions to be executed by processor 1204.Main memory 1208 might also be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 1204. Computing circuit 1200 might likewiseinclude a read only memory (“ROM”) or other static storage devicecoupled to bus 1202 for storing static information and instructions forprocessor 1204.

The computing circuit 1200 might also include one or more various formsof information storage mechanism 1210, which might include, for example,a media drive 1212 and a storage unit interface 1220. The media drive1212 might include a drive or other mechanism to support fixed orremovable storage media 1214. For example, a hard disk drive, a floppydisk drive, a magnetic tape drive, an optical disk drive, a CD or DVDdrive (R or RW), or other removable or fixed media drive might beprovided. Accordingly, storage media 1214 might include, for example, ahard disk, a floppy disk, magnetic tape, cartridge, optical disk, a CDor DVD, or other fixed or removable medium that is read by, written toor accessed by media drive 1212. As these examples illustrate, thestorage media 1214 can include a computer usable storage medium havingstored therein computer software or data. For example, one or morememory components may include non-transitory computer readable mediumincluding instructions that, when executed by the processor 1204, causethe computing circuit 1200 to perform one or more functionalitiesdescribed herein.

In alternative embodiments, information storage mechanism 1210 mightinclude other similar instrumentalities for allowing computer programsor other instructions or data to be loaded into computing circuit 1200.Such instrumentalities might include, for example, a fixed or removablestorage unit 1222 and an interface 1220. Examples of such storage units1222 and interfaces 1220 can include a program cartridge and cartridgeinterface, a removable memory (for example, a flash memory or otherremovable memory component) and memory slot, a PCMCIA slot and card, andother fixed or removable storage units 1222 and interfaces 1220 thatallow software and data to be transferred from the storage unit 1222 tocomputing circuit 1200.

Computing circuit 1200 might also include a communications interface1224. Communications interface 1224 might be used to allow software anddata to be transferred between computing circuit 1200 and externaldevices. Examples of communications interface 1224 might include a modemor softmodem, a network interface (such as an Ethernet, networkinterface card, WiMedia, IEEE 802.XX or other interface), acommunications port (such as for example, a USB port, IR port, RS232port Bluetooth® interface, or other port), or other communicationsinterface. Software and data transferred via communications interface1224 might typically be carried on signals, which can be electronic,electromagnetic (which includes optical) or other signals capable ofbeing exchanged by a given communications interface 1224. These signalsmight be provided to communications interface 1224 via a channel 1228.This channel 1228 might carry signals and might be implemented using awired or wireless communication medium. Some examples of a channel mightinclude a phone line, a cellular link, an RF link, an optical link, anetwork interface, a local or wide area network, and other wired orwireless communications channels.

In this document, the terms “computer program medium” and “computerusable medium” are used to generally refer to media such as, forexample, memory 1208, storage unit 1220, media 1214, and channel 1228.These and other various forms of computer program media or computerusable media may be involved in carrying one or more sequences of one ormore instructions to a processing device for execution. Suchinstructions embodied on the medium, are generally referred to as“computer program code” or a “computer program product” (which may begrouped in the form of computer programs or other groupings). Whenexecuted, such instructions might enable the computing circuit 1200 toperform features or functions of the disclosed technology as discussedherein.

While various embodiments of the disclosed technology have beendescribed above, it should be understood that they have been presentedby way of example only, and not of limitation. Likewise, the variousdiagrams may depict an example architectural or other configuration forthe disclosed technology, which is done to aid in understanding thefeatures and functionality that can be included in the disclosedtechnology. The disclosed technology is not restricted to theillustrated example architectures or configurations, but the desiredfeatures can be implemented using a variety of alternative architecturesand configurations. Indeed, it will be apparent to one of skill in theart how alternative functional, logical or physical partitioning andconfigurations can be implemented to implement the desired features ofthe technology disclosed herein. Also, a multitude of differentconstituent circuit names other than those depicted herein can beapplied to the various partitions. Additionally, with regard to flowdiagrams, operational descriptions and method claims, the order in whichthe steps are presented herein shall not mandate that variousembodiments be implemented to perform the recited functionality in thesame order unless the context dictates otherwise.

Although the disclosed technology is described above in terms of variousexemplary embodiments and implementations, it should be understood thatthe various features, aspects and functionality described in one or moreof the individual embodiments are not limited in their applicability tothe particular embodiment with which they are described, but instead canbe applied, alone or in various combinations, to one or more of theother embodiments of the disclosed technology, whether or not suchembodiments are described and whether or not such features are presentedas being a part of a described embodiment. Thus, the breadth and scopeof the technology disclosed herein should not be limited by any of theabove-described exemplary embodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; the terms “a” or“an” should be read as meaning “at least one,” “one or more” or thelike; and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “circuit” does not imply that the components or functionalitydescribed or claimed as part of the circuit are all configured in acommon package. Indeed, any or all of the various components of acircuit, whether control logic or other components, can be combined in asingle package or separately maintained and can further be distributedin multiple groupings or packages or across multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives can be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

What is claimed is:
 1. A system for remotely communicating instructions,the system comprising: a server communicatively coupled to a first userdevice and an instructor device, the server, the first user device, andthe instructor device individually comprising a memory and a processor,the server configured to: receive visual content captured by the firstuser device; transmit at least a portion of the captured visual contentto the instructor device, the instructor device configured to presentthe visual content within an instructor interface and receive an inputfrom an instructor, the input from the instructor defining aninstruction associated with the visual content; receive instructioninformation corresponding to the instruction associated with the visualcontent from the instructor device; and transmit at least a portion ofthe instruction information to the first user device, the first userdevice configured to present a dynamic object corresponding to theinstruction as an overlay on top of at least a portion of the visualcontent within a first learner interface based on the receivedinstruction information; wherein the dynamic object indicates how toperform the instruction, and wherein a changing color of the dynamicobject corresponds to a particular physical movement to be made toperform the instruction.
 2. The system of claim 1, wherein the firstlearner interface includes a record option to record one or moreportions of the instruction being presented on top of the visualcontent.
 3. The system of claim 1, wherein the first learner interfaceincludes a bookmark option to bookmark one or more portions of theinstruction being presented on top of the visual content.
 4. The systemof claim 1, wherein the instructor interface includes a segmentationoption to segment the presentation of the instruction by the first userdevice into multiple parts.
 5. The system of claim 4, whereinpresentation of different parts of the instruction by the first userdevice is controlled through the first learner interface or theinstructor interface.
 6. The system of claim 1, wherein the firstlearner interface includes a change option to change the instruction andthe server is further configured to facilitate exchange of the change tothe instruction between the first user device and the instructor device.7. The system of claim 1, wherein the presentation of the instruction bythe first user device includes a visual representation of a usage of anitem with respect to a target at the location.
 8. The system of claim 7,wherein the visual representation of the usage of the item with respectto the target at the location is scaled based on a size of the target.9. The system of claim 7, wherein the instructor interface includes anitem option to allow the instructor to interact with a visualrepresentation of the item to define the instruction on the usage of theitem with respect to the target at the location.
 10. The system of claim9, wherein the visual representation of the item is presented in theinstructor interface based on a determination that the item is availablefor use at the location.
 11. The system of claim 9, wherein the itemoption includes a set of preset options defining preset usage of theitem with respect to the target at the location.
 12. The system of claim11, wherein at least some of the set of preset options are includedwithin the instructor interface based on the visual content captured bythe first user device further defining one or more characteristics ofthe target at the location.
 13. The system of claim 9, wherein the itemoption allows the instructor to define the usage of the item withrespect to the target at the location based on a three-dimensional modelof the target.
 14. The system of claim 7, wherein the visualrepresentation of the usage of the item with respect to the target atthe location is presented based on a three-dimensional model of thetarget.
 15. The system of claim 14, wherein the three-dimensional modelof the target is generated based on a three-dimensional mapping of thetarget by the first user device, wherein the visual content captured bythe first user device further defines the three-dimensional mapping ofthe target.
 16. The system of claim 14, wherein: the server furthercommunicatively couples to a second user device; the server is furtherconfigured to transmit companion instruction information to the seconduser device, the companion instruction information defining theinstruction; and the second user device is configured to present thevisual representation of the usage of the item with respect to thetarget at the location within a second learner interface based on thereceived companion instruction information.
 17. The system of claim 16,wherein the presentation the visual representation of the usage of theitem with respect to the target at the location by the first user deviceand the second user device include different perspectives of the usageof the item with respect to the target at the location based onorientations of the first user device and the second user device withrespect to the target at the location.
 18. The system of claim 1,wherein the visual content includes one or more images or one or morevideos of the location.
 19. A method for remotely communicatinginstructions, the method performed by a server communicatively coupledto a first user device and an instructor device, the server, the firstuser device, and the instructor device individually comprising a memoryand a processor, the method comprising: receiving visual contentcaptured by the first user device; transmitting at least a portion ofthe captured visual content to the instructor device, the instructordevice configured to present the visual content within an instructorinterface and receive an input from an instructor, the input from theinstructor defining an instruction associated with the visual content;receiving instruction information corresponding to the instructionassociated with the visual content from the instructor device; andtransmitting at least a portion of the instruction information to thefirst user device, the first user device configured to present a dynamicobject corresponding to the instruction as an overlay on top of at leasta portion of the visual content within a first learner interface basedon the received instruction information; wherein the dynamic objectindicates how to perform the instruction, and wherein a changing colorof the dynamic object corresponds to a particular physical movement tobe made to perform the instruction.
 20. A non-transitory computerreadable medium comprising instructions that, when executed, cause aserver communicatively coupled to a first user device and an instructordevice to perform: receiving visual content captured by the first userdevice; transmitting at least a portion of the captured visual contentto the instructor device, the instructor device configured to presentthe visual content within an instructor interface and receive an inputfrom an instructor, the input from the instructor defining aninstruction associated with the visual content; receiving instructioninformation corresponding to the instruction associated with the visualcontent from the instructor device; and transmitting at least a portionof the instruction information to the first user device, the first userdevice configured to present a dynamic object corresponding to theinstruction as an overlay on top of at least a portion of the visualcontent within a first learner interface based on the receivedinstruction information; wherein the dynamic object indicates how toperform the instruction, and wherein a changing color of the dynamicobject corresponds to a particular physical movement to be made toperform the instruction.